Constraining the Size Growth of the Task Space with Socially Guided Intrinsic Motivation using Demonstrations

This paper presents an algorithm for learning a highly redundant inverse model in continuous and non-preset environments. Our Socially Guided Intrinsic Motivation by Demonstrations (SGIM-D) algorithm combines the advantages of both social learning and intrinsic motivation, to specialise in a wide range of skills, while lessening its dependence on the teacher. SGIM-D is evaluated on a fishing skill learning experiment.Comment: JCAI Workshop on Agents Learning Interactively from Human Teachers (ALIHT), Barcelona : Spain (2011

Bootstrapping Intrinsically Motivated Learning with Human Demonstrations

This paper studies the coupling of internally guided learning and social interaction, and more specifically the improvement owing to demonstrations of the learning by intrinsic motivation. We present Socially Guided Intrinsic Motivation by Demonstration (SGIM-D), an algorithm for learning in continuous, unbounded and non-preset environments. After introducing social learning and intrinsic motivation, we describe the design of our algorithm, before showing through a fishing experiment that SGIM-D efficiently combines the advantages of social learning and intrinsic motivation to gain a wide repertoire while being specialised in specific subspaces.Comment: IEEE International Conference on Development and Learning, Frankfurt : Germany (2011

R-IAC : Robust Intrinsically Motivated Active Learning

International audienceIAC was initially introduced as a developmental mechanisms allowing a robot to self-organize developmental trajectories of increasing complexity without pre-programming the particular developmental stages. In this paper, we argue that IAC and other intrinsically motivated learning heuristics could be viewed as active learning algorithms that are particularly suited for learning forward models in unprepared sensorimotor spaces with large unlearnable subspaces. Then, we introduce a novel formulation of IAC, called R-IAC, and show that its performances as an intrinsically motivated active learning algorithm are far superior to IAC in a complex sensorimotor space where only a small subspace is neither unlearnable nor trivial. We also show results in which the learnt forward model is reused in a control scheme

Maturationally-Constrained Competence-Based Intrinsically Motivated Learning

International audienceThis paper studies the coupling of intrinsic motiva- tion and physiological maturational constraints, and argues that both mechanisms may have complex bidirectional interactions allowing to actively control the growth of complexity in motor development. First, we introduce the self-adaptive goal generation algorithm (SAGG), instantiating an intrinsically motivated goal exploration mechanism for motor learning of inverse models. Then, we introduce a functional model of maturational con- straints inspired by the myelination process in humans, and show how it can be coupled with the SAGG algorithm, forming a new system called McSAGG. We then present experiments to evaluate qualitative properties of these systems when applied to learning a reaching skill with an arm with initially unknown kinematics

Intrinsically motivated exploration as efficient active learning in unknown and unprepared spaces

International audienceIntrinsic motivations are mechanisms that guide curiosity-driven exploration (Berlyne, 1965). They have been proposed to be crucial for self-organizing developmental trajectories (Oudeyer et al. , 2007) as well as for guiding the learning of general and reusable skills (Barto et al., 2005). Here, we argue that they can be considered as “active learning” algorithms, and show that some of them also allow for very efficient learning in unprepared sensorimotor spaces, outperforming existing active learning algorithms

Intrinsically Motivated Goal Exploration for Active Motor Learning in Robots: a Case Study

International audienceWe introduce the Self-Adaptive Goal Generation - Robust Intelligent Adaptive Curiosity (SAGG-RIAC) algorithm as an intrinsically motivated goal exploration mechanism which allows a redundant robot to efficiently and actively learn its inverse kinematics. The main idea is to push the robot to perform babbling in the goal/operational space, as opposed to motor babbling in the actuator space, by self-generating goals actively and adaptively in regions of the goal space which provide a maximal competence improvement for reaching those goals. Then, a lower level active motor learning algorithm, inspired by the SSA algorithm, is used to allow the robot to locally explore how to reach a given self-generated goal. We present simulated experiments in a 32 dimensional continuous sensorimotor space showing that 1) exploration in the goal space can be a lot faster than exploration in the actuator space for learning the inverse kinematics of a redundant robot; 2) selecting goals based on the maximal improvement heuristics is statistically significantly more efficient than selecting goals randomly

The effects of task difficulty, novelty and the size of the search space on intrinsically motivated exploration

International audienceDevising efficient strategies for exploration in large open-ended spaces is one of the most difficult computational problems of intelligent organisms. Because the available rewards are ambiguous or unknown during the exploratory phase, subjects must act in intrinsically motivated fashion. However, a vast majority of behavioral and neural studies to date have focused on decision making in reward-based tasks, and the rules guiding intrinsically motivated exploration remain largely unknown. To examine this question we developed a paradigm for systematically testing the choices of human observers in a free play context. Adult subjects played a series of short computer games of variable difficulty, and freely choose which game they wished to sample without external guidance or physical rewards. Subjects performed the task in three distinct conditions where they sampled from a small or a large choice set (7 vs. 64 possible levels of difficulty), and where they did or did not have the possibility to sample new games at a constant level of difficulty. We show that despite the absence of external constraints, the subjects spontaneously adopted a structured exploration strategy whereby they (1) started with easier games and progressed to more difficult games, (2) sampled the entire choice set including extremely difficult games that could not be learnt, (3) repeated moderately and high difficulty games much more frequently than was predicted by chance, and (4) had higher repetition rates and chose higher speeds if they could generate new sequences at a constant level of difficulty. The results suggest that intrinsically motivated exploration is shaped by several factors including task difficulty, novelty and the size of the choice set, and these come into play to serve two internal goals—maximize the subjects' knowledge of the available tasks (exploring the limits of the task set), and maximize their competence (performance and skills) across the task set

Eye movements reveal epistemic curiosity in human observers

International audienceSaccadic (rapid) eye movements are primary means by which humans and non-human primates sample visual information. However, while saccadic decisions are intensively investigated in instrumental contexts where saccades guide subsequent actions, it is largely unknown how they may be influenced by curiosity – the intrinsic desire to learn. While saccades are sensitive to visual novelty and visual surprise, no study has examined their relation to epistemic curiosity – interest in symbolic, semantic information. To investigate this question, we tracked the eye movements of human observers while they read trivia questions and, after a brief delay, were visually given the answer. We show that higher curiosity was associated with earlier anticipatory orienting of gaze toward the answer location without changes in other metrics of saccades or fixations, and that these influences were distinct from those produced by variations in confidence and surprise. Across subjects, the enhancement of anticipatory gaze was correlated with measures of trait curiosity from personality questionnaires. Finally, a machine learning algorithm could predict curiosity in a cross-subject manner, relying primarily on statistical features of the gaze position before the answer onset and independently of covariations in confidence or surprise, suggesting potential practical applications for educational technologies, recommender systems and research in cognitive sciences. With this article, we provide full access to the annotated database allowing readers to reproduce the results. Epistemic curiosity produces specific effects on oculomotor anticipation that can be used to read out curiosity states

Incremental Local Online Gaussian Mixture Regression for Imitation Learning of Multiple Tasks

International audienceGaussian Mixture Regression has been shown to be a powerful and easy-to-tune regression technique for imitation learning of constrained motor tasks in robots. Yet, current formulations are not suited when one wants a robot to learn incrementally and online a variety of new context- dependant tasks whose number and complexity is not known at programming time, and when the demonstrator is not allowed to tell the system when he introduces a new task (but rather the system should infer this from the continuous sensorimotor context). In this paper, we show that this limitation can be addressed by introducing an Incremental, Local and Online variation of Gaussian Mixture Regression (ILO-GMR) which successfully allows a simulated robot to learn incrementally and online new motor tasks through modelling them locally as dynamical systems, and able to use the sensorimotor context to cope with the absence of categorical information both during demonstrations and when a reproduction is asked to the system. Moreover, we integrate a complementary statistical technique which allows the system to incrementally learn various tasks which can be intrinsically defined in different frames of reference, which we call framings, without the need to tell the system which particular framing should be used for each task: this is inferred automatically by the system

A new experimental setup to study the structure of curiosity-driven exploration in humans

International audienceWe are presenting an exploratory study with humans designed to analyze and measure properties of curiosity-driven exploration of a priori unknown sensorimotor spaces. More specifically, we are interested in the relation between exploration and learning progress